On Match Lengths and the Asymptotic Behavior of 
Sliding Window Lempel-Ziv Algorithm for Zero 

Entropy Sequences 
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Abstract— The Sliding Window Lempel-Ziv (SWLZ) algorithm 
has been studied from various perspectives in information theory 
literature. In this paper, we provide a general law which defines 
the asymptotics of match length for stationary and ergodic zero 
entropy processes. Moreover, we use this law to choose the match 
length L in the almost sure optimality proof of Fixed Shift 
Variant of Lempel-Ziv (FSLZ) and SWLZ algorithms given in 
literature. First, through an example of stationary and ergodic 
processes generated by irrational rotation we establish that for 
a window size of n w a compression ratio given by 0( '° B "™ ) 
where a is arbitrarily close to 1 and < a < 1, is obtained 
under the application of FSLZ and SWLZ algorithms. Further, 
we give a general expression for the compression ratio for a class 
of stationary and totally ergodic processes with zero entropy. 

I. Introduction 

The analysis of the string matching universal data compres- 
sion algorithm in probabilistic setting was initiated by Wyner 
and Ziv iflOl using the results from ergodic theory. This work 
was followed by the results on return times for stationary 
and ergodic sources by Ornstein and Weiss 0. Further, 
connections between return times and data compression have 
been obtained by Kontoyiannis [4J. Kim and Park |2J have 
given results on return time for zero entropy ergodic processes 
generated by irrational rotation. 

The Sliding Window Lempel Ziv (SWLZ) algorithm, which 
is very similar to the original string matching algorithm, 
was introduced by Wyner and Ziv |[[ 1 j and its asymptotic 
optimality for the class of stationary and ergodic sources was 
proved in expected sense. Shields [9| proved the optimality 
of SWLZ algorithm for individual sequences by comparing 
the number of phrases of the SWLZ algorithm with that of 
the incremental parsing algorithm LZ-78 0~3]. On similar 
lines as Wyner and Ziv ifTTl . Jacob and Bansal [1| proved 
the asymptotic optimality of the SWLZ algorithm in almost 
sure sense. However, it is true that the match length L a used 
by Wyner and Ziv and Jacob and Bansal can be used to 
prove the almost sure optimality of SWLZ algorithm for zero 
entropy processes but in this paper motivated by the behavior 
of SWLZ on periodic sequences (which is discussed in detail 
in section II) and recent results by Kim and Park J2) on 
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recurrence properties for irrational rotations and in general for 
almost every interval exchange map with zero entropy ||3], 
we show that for a general zero entropy process as well, 
it is possible to choose a larger match length L a in the 
proof of optimality of SWLZ algorithm, through which we 
establish results on convergence rate of the compression ratio 
of the SWLZ algorithm that are achievable for zero entropy 
sequences. 

Lastras-Montano 01 showed that for a positive entropy 
finite order aperiodic and irreducible Markov source, SWLZ 
algor ithm achi eves a compression ratio of the order of H + 
0(\J ^£ °f l n " w ) for a window of size n w . In this paper, 
we obtain a finer statement for zero entropy cases. This is 
discussed in detail in section III and IV. 

The remainder of this paper is organized as follows. In 
Section III we provide results on return times and match 
lengths and establish a theorem on match length. In Section 
IV subsections A and B, we prove the almost sure optimality 
of FSLZ and SWLZ algorithms respectively by making a 
different choice of match length L for zero entropy processes 
as compared to the one chosen by Wyner and Ziv in IfTTl or 
Jacob and Bansal in [1 1. Further, this is followed by providing 
tighter upper bounds on convergence rate of the compression 
ratio for a class of zero entropy processes for these algorithms. 
In Section V, we present our conclusion. 

II. Motivation 

Consider x = 010101..., and Tx = 101010 , to be two n 

long sequences. Let P(x) = P(Tx) — 0.5 be the probability 
of occurence of x and Tx respectively. So, {x, Tx} define 
the 2 realizations of a stationary ergodic process which are 
periodic and entropy rate of the process is zero. 

If LZ 78 is applied on the sequence x then the the number 
of phrases generated are given by c(n) ~ for an n long 
segment of the sequence x, which gives the compression ratio 

b c(n)logc(»)+c(») _ njjogn+nj _ logn However jf 

SWLZ algorithm is applied on the sequence x, 1 bit will be 
required to specify the start in the initial window of size 2 



and logn bits will be required to specify the length of the 
match, since in one iteration complete match will be found 
in the initial window itself. So, in this case the compression 
ratio will be which converges faster as compared to the 

compression ratio in the case of LZ-78 applied on sequence x. 
This motivates the study of the behavior of SWLZ algorithm 
on aperiodic sequences generated by an ergodic process of 
zero entropy rate. 

III. Return Times and Match Lengths 
Let the sequence x = {in}_oo be an instance of an ergodic 
source X = {Xn}^^ w i m finite alphabet A. Here, x J k denotes 

the segment Xk,Xk+i, , Xj for sequence x. For n > 0, 

define 

Rn(x) = mm{l : I > 0,x? = aCj^}. 
Thus, R n (x) is the first return time of the word x" in the past. 



L n (x) = max{j : j > 0, x\ 



-k+j 

-k+V 1 



1,2, ....,n}. 



Thus L n (x) is the length of the longest possible match in the 
previous n symbols x°_ n+1 . On the asymptotic behavior of 
Rn{x) and L n {x) we have the following, 
Theorem 1 (Ornstein and Weiss): With probability 1, for a 
stationary and ergodic source X 

lim l0gRn{x) = H(X); lim ^ = H(X). 

n— s-oo n n— s-oo L n (x) 

where H(X) is the entropy of the source X. 

As a general case, consider a stationary and ergodic process 

for which the asymptotics of the first return time follow 



logi?„(x) 

llm — 7T~\ — = c ' 
j(n) 



(1) 



where f(n) < n is an invertible and increasing function of 
n and c > is a constant. It is notable here that for a zero 
entropy process f(n) = o(n). 

Theorem 2: With probability 1 for a stationary and ergodic 
process satisfying Eq. (|TJ 

logn 
lim — — — — = c. 

n^oo f(L n (x)) 

Proof of Theorem 2: Let n and m be natural numbers such 
that n = L m (x). Hence, by definition of a match length we 
have 

R n +i(x) > m and R n {x) < m. (2) 
Therefore, using Inequality © we have 



logi? n+ i(x) logm > log Rn(x) 



(3) 



/(n) ' f(L m (x)) ~ f(n) 

Now we apply the limit n —> oo on inequality (01 and hence 
using Sandwich theorem and making a change in the name 
of variable from m to n we have obtained the statement of 
Theorem 2. 

Now we shift our attention to zero entropy sources. For 
this purpose, we first consider the sources generated by an 
irrational rotation. 



Let v e 1Z and let ||w|| denote its distance to the nearest 
integer, i.e., ||u|| = mm ne z \v — n\. 

Let 8 be an irrational number in (0,1). Consider T : 
[0, 1) — > [0, 1) to be an irrational rotation by 8. i.e., T(x) = 
(x + 8) mod 1. T preserves the Lebesgue measure on 
X = [0, 1). For an irrational 8 e (0, 1), define n(8) = sup{t > 
: lim inf j^oo = 0}. In general, T)(8) > 1 and = 1 for 

almost every 8 121 ■ For generating a sequence x™ by irrational 
rotation a partition V = {E, E c } is defined on [0, 1], such that 
Xi = if T l ~ x x e E and 1 otherwise. 
Theorem 3 (Kim and Park): For almost every sequence 
generated by rotation by an irrational 8 e (0, 1): 



lim inf 



log R n {x) 
log n 



1 \ogR n (x) 
- ; lim sup — ; = 1 . 



(4) 



r](9)' n^oo^ logn 

which implies that if 9 is such that r](d) — 1 then the limit 
exists and is given by: 



lim l ° gRn{x) = 1. 

rwoo logn 



(5) 



Now, applying Theorem 2 by choosing the function /(n) = 
logn, we immediately deduce the following corollary 
Corollary 1: For a stationary and ergodic process generated 
by irrational rotation where tj(6) = 1 with probability 1, 

llm log" _ 1 

Hindoo logL„(a:) ~ l ' 

Critical Observation 1: Consider a stationary and ergodic 
process satisfying the law given by Eq. (|TJ 

By Eq. (Q} and Egoroff's Theorem [7], for a given S > 
and e > 0, 3 a set i*\ of sequences x with P{Fi) > 1 — 5 and 
an integer N (e,F±) s.t. 



Q< \o g R n (x) <c + e 



f(n) 

log Rn(x) 



< f(n) R n (x) < 2^^ n \ 



Now, if n w is the window size and n w = 2^ c+e ^^ Lo \ then 
we have R^ o (x) < n w which implies that a match of length 
L a is bound to be found in n w symbols and hence the match 
length is given by 

L =[f-\^)}. (6) 

For example in a positive entropy case, where /(n) = n and 
c = H, L a = I l0 ft"™ J and in case of irrational rotation where 
as given by Eq. Q in Theorem 3, f(n) = logn, c = 1 and 
rj{8) = 1, we have 



Lo = \n w x + e J. 



(7) 



IV. Convergence Rate of the Compression Ratio 
It has bee n proved in [6| that the compression ratio given 
by ff + 0(y / ^%|^) is achieved by SWLZ algorithm for an 
aperiodic and irreducible Markov source with positive entropy. 
Following the discussion in Section EH, in this section we 
prove that a compression ratio given by 0( *?"" ), where < 
a < 1, is achieved by Fixed Shift Variant of SWLZ (FSLZ) 



algorithm [ 1 1 and SWLZ algorithm for stationary and ergodic 
processes generated by an irrational rotation. Further, we give 
a general expression of the compression ratio for zero entropy 
cases under the setting described by Eq. (0 for both FSLZ 
and SWLZ algorithms. 

In the remainder of this section, we illustrate through the 
example of stationary and ergodic processes generated by 
irrational rotations that a compression ratio given by 0( 1 ° s ""' ) 
where < a < 1 and a is arbitrarily close to 1, is achieved by 
FSLZ and SWLZ algorithms respectively, following the proofs 
of optimality of these algorithms as given in ifTTI and fl] but 
choosing L given by Eq. (0. It is emphasized here that the 
L Q chosen is in contrast to what is chosen in [11 1 and (TJ. 
Moreover, following this illustration, a general expression for 
the compression ratio for a class of zero entropy stationary and 
totally ergodic processes is given under a certain restriction on 
the convergence rate of the law given in Eq. (0. 

A. Fixed Shift Variant of SWLZ Algorithm (FSLZ) 

We first consider FSLZ algorithm introduced in [1| to gain 
insight into the working of the universal SWLZ algorithm. Let 
us consider the string x^ consisting of the first N symbols of 
the sequence. Following the discussion in Section III. Using 
critical observation 1, let us take for e > 0, the match 
length given by Eq. 0. Note the contrast in the choice of 
match length that is used in [ 1 1 1 and 1 1 1 for positive entropy 
stationary and ergodic processes. The algorithm is described as 
follows: 

1) Initialization: A window of size n w is fixed. Let j = 1 
and rij = n w . The first n w symbols are transmitted 
without compression. Let x™ 1 be the current window. 

2) Matching: At the jth step if rij + L Q > N, terminate 



the algorithm, else if 



any fc e 1,2, .. 



-n w +k+L 



for 

then there is a match, i.e., sequence 



° has a match in the current window. 

3) Coding: If there is a match, let Sj be the location of the 
match in the current window. Then [logn TO ] bits are 
required to code Sj, If a match is not found the number 
of bits required is f3L Q . In addition, a one bit flag is used 
to specify if the match is found or not found. 

4) Sliding: rij+i = rij + L Q , i.e., the current window now 
becomes a;™^'_ n „ +1 . 

Repeat the steps 2, 3 and 4 until the sequence xf terminates 
at the (m + l)-th step. The remaining N — n w — mL < L a 
symbols are encoded without compression. Thus, a total of 
N — mL < n w + L a are encoded without any compression. 
Let mi be the number of blocks that have a match and m 2 
be the number of blocks that do not have a match. One bit 
per block is used to denote if it is good or bad. Hence, taking 
overheads into account, the number of bits per symbol required 
by the code is given by 

Rfslz = tf (N - mL )(3 + mi [log n w ] + m 2 (3L + m 



< 



N 
1 

N 



(n w + L )j3 + mi [log n w ] + m 2 /3L a + m 



(8) 



The first term in the expression converges to as N —> oo. 
Also, we have 

N 

mi <m< — . (9) 

L 



Using Eq. and (0 we have, 

mi [log n w ] [log n w ] 
N L n 



[log n w ] 



n,„ i+« 



1 



(10) 



which converges to as n w — > oo. Therefore, using Eq. (fTOt 
we have 



lim ^=0, 



(11) 



Hence, we have the following 

Critical Observation 2: The second term in the expression 
of Rfslz (Eq. (EJl) converges to for all e > as n w — > oo 
and the convergence rate is given by 0( 



log n 



Now , define G = {x : Rl [x) > n w }. Thus, if x e G, then 
x does not have a match in the previous n w symbols. We use 
1g to denote the indicator function of G. Since T is totally 
ergodic (i.e., T k is ergodic for all k as k9 is irrational for all 
fc), Z the shift transformation is also totally ergodic. By the 
Ergodic Theorem (HU we have (almost surely) 



m 2 L m 2 
N m 



£7=o W- 



(*)) 



KG)- (12) 



Now, 



G = {x : Rl (x) > n w } 

c{x: l ^M>l + e}. 
logL 

Hence, by Theorem 3 lirrin^^oo /i(G) = 0. 
Critical Observation 3: It is established in ||2] Sec. 2 page- 
3944, Sec. 3 page-3948, 3949] that the convergence rate of 
the probability of the set G is atleast as fast as — i^, where 
b > a, for a defined in result 1 and result 3 below. 

Combining ©, ©, (Qj), CE]> and fi(G) -> 0, we have 
for almost every sequence generated by an irrational ro- 
tation, the FSLZ algorithm is asymptotically optimal, i.e., 

li m n„^oo liniAr_).oo Rfslz = 0. 

Using critical observations 2 and 3, we can now state the 
following 

Result 1: For a stationary and ergodic process generated by 
an irrational rotation a compression ratio given by 0( *?""* ) 
is achieved by FSLZ algorithm, where a — — and e > is 
arbitrary. 

Now, we consider a general setting where the process is 
assumed to be stationary and totally ergodic with zero entropy 
and the asymptotics of the first return time follow the law 
given by Eq. (01. In such a case following the proof of FSLZ 
algorithm considered above, we choose L Q given by Eq. (|6]). 
Imitating the proof given above with this choice of L Q yields 



us a term of the form — J 1 ?,? "'"I 



\o S R Lo (x) 



jn^ — > c + e}. Since f(n) = o( 
[log n w ] 



- and the set G c {a 
we have 



lim 



log n u 
c+e 



)-i 



o. 



(13) 



Also, using Eq. (Q3, lim^^go 11(G) = 0. 
If we further assume 

Assumption 1: The rate of convergence of /i(G) to 0, is at 



least as fast as 



, then we have 



Result 2: For a stationary and totally ergodic process for 
which the asymptotics of first return time follow the law given 
by Eq. (fl3 and assumption 1 holds, a compression ratio given 
-) is achieved by FSLZ algorithm. Here e > 



log n v 



is arbitrary. 

For example, if f(n) = ^/n the compression ratio for FSLZ 
algorithm is given by Q( [° „ )■ 

B. The Sliding Window Lempel-Ziv Algorithm 

Consider a sequence x — {x n }f which is sequentially 
made available to the encoder. Let A be the finite set of 
alphabet for the sequence x. Let (3 — [log \ A\~\ . If S is a finite 
set then |5| denotes the cardinality of S. Sliding Window LZ 
is now described ifTTl (T) : 

Consider the string x 1 ^ consisting of the first N symbols of 
the sequence x — {x n }^°. The algorithm works as follows: 

1) Initialization: A window of size n w is fixed. Let j = 1 
and rij = n w . The first n w symbols are transmitted 
without compression. Let x™ 1 be the current window. 

2) Matching: At the jth step let Lj be the largest integer 

71 I Jj 

such that the copy of x n ™ +1 3 begins in the current 
window and rij + Lj < N. Let Sj denote the starting 
index of the match in the current window. The matched 
phrase x"™^ J is denoted by x^\ If a match is not 



found Li 



1, Sj 



0. 



3) Coding: If Sj > 0, the length Lj of the matched code can 
be specified using -f\og(Lj + 1) bits using the integer 
code described in IfTTl . The matched location Sj can 
be specified using [logn^] bits. If Sj = or if, using 
the above procedure, the total number of bits needed to 
represent a phrase exceeds (3Lj then fiLj bits are used 
for encoding. A one bit flag is used to denote which of 
these two encoding schemes was used. 

4) Sliding: For the next window rij + i = rij + Lj and the 
window for the next iteration is x™ 3+1 _„ , -, . 

** j + 1 ' ''in 1 J- 

Repeat the steps 2,3 and 4 until the sequence xf is exhausted. 

Let B(x^')+1 denote the number of bits required to encode 
the j-th phrase. Then, we have 

B{x&) = min{ 7 log( J L j + 1) + [log^],^}. (14) 

If the total number of phrases is c(N) then, the number of 
bits required per symbol is: 



Rs 



WLZ 



1 r 

N 



n, 



c(N) 

,P+ B{x^) + c(N) 



(15) 



Here, we analyze the performance of SWLZ algorithm on zero 
entropy sequences generated by irrational rotation Transfor- 
mation T. The idea behind the analysis is inspired from the 
method of partitioning of sequences used by IfTTl . However, 
unlike them the class of partitions considered here are obtained 



by shifting the partition used by them. By this the optimality 
result is obtained in almost sure sense JTJ. Consider the inter- 
vals defined as I r = [r + n w , r+n w +L — 1], r = 1,2, N'. 
Here, N' = N — L a — n w + 1. Interval I r is bad if a copy 
of (xi) ieI does not begin in the string n w — L a symbols 
preceeding it. Let m be the number of such bad intervals. 
Now, we define G = {x : Rl (x) > n w — L a }. So , if x e G, 
then [l,L ] is a bad interval. By Ergodic Theorem almost 
surely, 



m to 
N < N 7 



Now, 



N' 

G = {x : Rl o (x) > n l 



H(G). {as N -)• 00) 
(16) 

-L } 



= {x: 
Hence by Theorem 3, 



C {x : R Lo (x) > L +2 } 
log R Lo (x) 



log L 



>l+oi- 



lim n{G) = 0. 



(17) 



which implies from ( TToT l linv^^oo S = 0. As, already 
specified in critical observation 3, we have the convergence 
of rate of 11(G) to (as n w — > 00) is at least as fast as ^-r, 
where b > a. 

Let us consider the SWLZ parsing of the sequence Xi . A 
phrase is defined to be internal with respect to interval I r if it 
begins and ends in the interval [r + n w , r + L Q + n w — 2] [ 1 1 1 . 
Moreover, an interval I r with an internal phrase is bad IfTTl . 

For k = 0,1,2 ,L Q — 1 , a partition Vk is defined 

as given in JJJ Section V]. Let the collection of internal 
phrases corresponding to Vk together with initial and final 
segment be denoted by set Sk and let rrik be the number of 
bad intervals. Then, using bound on J2j t s k an( l defining 
77ifc o = mirifclm/c} as in UJ Section V], L m,k < m is 
obtained. 

Using Eq. (1151 . the number of bits per symbol for SWLZ 
algorithm is given by 



Rs 



1 

U LZ = Jj 



n w f3 - 



1) 



- E ^ 



Rswlz < 



+ 2L Q )(P + 1) + (f3 + l)m 



■U)) 



\Si 



1 



(18) 



The terms V^di+ll anc j ^vp+^)-^o convel -g e to as JV -> cxd. 
From Eq. ( TTTb it is evident that the second term in the 
expression for Rswlz given by Eq. ( fT8l converges to (as 
N — > 00 followed by n w — > 00). Now, we consider the 
last term in the expression for Rswlz given in Eq. (TT~8b 
i- e -> ~k^jeS% B(x^). Using the method given in ifTTIl . 
let d = \S)j. \. Since, comprises of non internal phrases 
corresponding to partition Vk a together with the initial and 



2(/3+l)L 



final segments, the non-internal phrase should end at the last 
index of any interval that belongs to Vk ■ 



N' N 



SZ =d< — < 



L L 



Result 4: For a stationary and totally ergodic process with zero 
entropy such that the asymptotics of its first return time follow 
the law given by Eq. (fl]i with f(n) > logn and assumption 1 

log — — ) is achieved 



(19) holds a compression ratio given by 0{ — 



>oo N 



0. Further, 



Using Eq. $1% we have lim„ u 
imitating the method used in ifTTl . we have 

i J2 B( X W) = i {-rlog(ii + 1) + riogn^l} 

= ±\Sl\\logn w ] + ± J2 7log(^ + l) 



by SWLZ algorithm. For example, if fOn) = \/n, the 

^3 



N ,1 . 
<I^ + f log(L + l) (a l} 
= 0. (as n w — > oo ) 

(20) 

Here, (ai) means using Eq. ( [T9i > and (02) means using the 
concavity of log function. 

Critical Observation 4: It is evident in Eq. ( f20l > that the com- 
pression ratio converges to for good phrases at the rate given 
by 0( S " M ), for every e > 0. Hence, combining results of Eq. 

dTTb and" ( f20b we get for almost every sequence generated by 
an irrational rotation, the SWLZ algorithm is asymptotically 
optimal i.e., lim„ ii ,_ s . 00 limjv->oo Rswlz{x) — a.s. 
So using critical observations 3 and 4 we have 
Result 3: For stationary and ergodic processes generated by 
an irrational rotation a compression ratio given by 0( 1 °f n ™ ) 
is achieved by SWLZ algorithm, where a = and e > is 
arbitrary. 

Now, we consider a general setting where the process is 
assumed to be stationary and ergodic with zero entropy and the 
asymptotics of the first return time follow the law given by Eq. 
(HJ. In such a case following the proof of SWLZ considered 
above, we choose L Q given by Eq. ©. Imitating the proof 
given above with this choice of L a yields us terms of the 
form . + f- log(L + 1) and the set 

n r log -RLq(z) ^ , e 

Gc{x: Hv^T >c+ 2 } - 



Since f(n) = o{n) 

[log n w ] 



lim 



) - 1 L 



\og(L + 1) = 0. (21) 



If f(n) > logn, then L Q < n w for every e > 0, the 
compression ratio is given by 0{ '^0""^ ) f° r every e > 0. 

Also we have using Eq. ([T]i Mm^^oo lAG) — 0. Hence, we 
have the following 



Here e > is 



compression ratio is given by 
arbitrary. 

V. Conclusion 

In this paper, we state Theorem 2 on match length which 
generalizes the match length result given by Omstein and 
Weiss for stationary and ergodic processes. Next, for the 
class of zero entropy processes we establish through corollary 
1, the behavior of match length asymptotics for irrational 
rotation and a general zero entropy stationary and ergodic case. 
Further, we imitated the proofs given in |fTT] and JT] of FSLZ 
and SWLZ algorithms by choosing a L a given by Eq. (O 
in contrast to their choice and showed that these algorithms 
achieve faster convergence rate of the compression ratio for 
zero entropy sequences as compared to those with positive 
entropy. It will be an interesting problem to look for totally 
ergodic processes that display the behavior given by Eq. ([T) 
where the function f(n) is different from logn. Also, it will 
be of interest to determine the class of irrational rotations 
and partitions V used to generate the sequences for which 
compression ratio converges to zero, uniformly. 
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